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DETAILED ACTION 

1 . Claims 1 - 24 are pending in this application. 

2. Claims 4 - 6, 8 - 1 0, 1 5 - 21 , 23 and 24 have been amended. 

Priority 

3. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 

Information Disclosure Statement 

4. The information disclosure statements (IDS) filed on 08/24/2006, 08/25/2006, 
07/18/2008 and 07/18/2008 are in compliance with the provisions of 37 CFR 1 .97, and 
have been considered and a copy is enclosed with this office action 

Claim Objections 

5. Claims 1 , 1 1 , 1 2, 22 - 24 objected to because of the following informalities: 
Claims 1,11,12 and 22 - 24 recite the limitation "generating a still image from a plurality 
of frame images contained in a video image", a video image, as it is commonly 
understood, refers to a single frame (a single image), therefore it cannot contain a 
plurality of frames. Examiner has assumed that instead of "from a video image" what 
was meant was "from a video sequence". Appropriate correction is required. 
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Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

7. Claims 1 - 1 0 are rejected under 35 U.S.C. 1 01 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 1 defines a "device". 
However, while the preamble defines a "device", which would typically be indicative of 
an "apparatus", the body of the claim lacks definite structure indicative of a physical 
apparatus. Furthermore, the specification on page 10 lines 5 - 8 indicates that the 
invention may be embodied as pure software. Therefore, the claim as a whole appears 
to be nothing more than a collection of software modules, thus defining functional 
descriptive material per se. Claims 2 - 10 fall accordingly because of their dependence 
on claim 1 . 

Functional descriptive material may be statutory if it resides on a "computer- 
readable medium or computer-readable memory". The claim(s) indicated above lack 
structure, and do not define a computer readable medium and are thus non-statutory for 
that reason (i.e., "When functional descriptive material is recorded on some computer- 
readable medium it becomes structurally and functionally interrelated to the medium 
and will be statutory in most cases since use of technology permits the function of the 
descriptive material to be realized" - Guidelines Annex IV). The scope of the presently 
claimed invention encompasses products that are not necessarily computer readable, 



Application/Control Number: 10/541,479 Page 4 

Art Unit: 2624 

and thus NOT able to impart any functionality of the recited program. The examiner 
suggests: 

1 . Amending the claim(s) to embody the program on "computer-readable 
medium" or equivalent; assuming the specification does NOT define the computer 
readable medium as a "signal", "carrier wave", or "transmission medium" which are 
deemed non-statutory; or 

2. Adding structure to the body of the claim that would clearly define a 
statutory apparatus. 

Any amendment to the claim should be commensurate with its corresponding 
disclosure. 

Note: 

"A transitory, propagating signal ... is not a "process, machine, manufacture, or 
composition of matter." Those four categories define the explicit scope and reach of 
subject matter patentable under 35 U.S.C. § 101 ; thus, such a signal cannot be 
patentable subject matter." (In re Petrus A.C.M. Nuijten; Fed Cir, 2006-1371 , 
9/20/2007). 

Should the full scope of the claim as properly read in light of the disclosure 
encompass non-statutory subject matter such as a "signal", the claim as a whole would 
be non-statutory. Should the applicant's specification define or exemplify the computer 
readable medium or memory (or whatever language applicant chooses to recite a 
computer readable medium equivalent) as statutory tangible products such as a hard 
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drive, ROM, RAM, etc, as well as a non-statutory entity such as a "signal", "carrier 
wave", or "transmission medium", the examiner suggests amending the claim to include 
the disclosed tangible computer readable storage media, while at the same time 
excluding the intangible transitory media such as signals, carrier waves, etc. 

8. Claim 23 and 24 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non-statutory subject matter as follows. Claims 23 and 24 are drawn to 
functional descriptive material recorded on a computer readable medium. Normally, the 
claim would be statutory. However, the specification, at page 10 defines or exemplifies 
the claimed computer readable medium as encompassing statutory media such as a 
"ROM", "hard drive", "optical drive", etc, as well as non-statutory subject mater such as 
a "data signal embodied in a carrier wave" . 

"A transitory, propagating signal ... is not a "process, machine, manufacture, or 
composition of matter." Those four categories define the explicit scope and reach of 
subject matter patentable under 35 U.S.C. § 101 ; thus, such a signal cannot be 

patentable Subject matter." ( In re Nuiiten. 84 USPQ2d 1495 (Fed, Cir. 2007) ). 

Because the full scope of the claim as properly read in light of the disclosure 
appears to encompass non-statutory subject matter (i.e., because the specification 
defines/exemplifies a computer readable medium as a non-statutory signal, carrier 
waver, etc.) the claim as a whole is non-statutory. The examiner suggests amending 
the claim to include the disclosed tangible computer readable storage media, while at 
the same time excluding the intangible transitory media such as signals, carrier waves, 
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etc. Any amendment to the claim should be commensurate with its corresponding 
disclosure. 

Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

10. Claims 1 -8, 11- 1 9 and 22 - 23 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Peleg et al., WO 98/02844 published on Jan. 22, 1998 ("Peleg"). 

1 1 . Regarding claim 1 , Peleg discloses 'An image generating device for generating a 
still image from a plurality of frame images contained in a video image' (see page 3, line 
32 - 36 ; page 5, lines 10 - 15). Peleg specifically suggests an apparatus for 
constructing an image mosaic (a still image from a plurality of images) and that the 
source of images can be a video camera (hence a video). 

'a synthesis object setting module for setting, from among areas included in 
frame images other than a reference frame image selected from among the plurality of 
frame images (see page 3 lines 28 - 36; Peleg specifically suggests a means for 
selecting source images and a means for selecting source image segments from the set 
of images ), one or more areas as object frame image areas for synthesis (see page 3, 
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lines 28 - 36; page 6 lines 7-10; Peleg specifically suggests cutting sub-images out of 
larger images to merged into the mosaic), the object frame image areas being selected 
according to a predetermined rule relating to a reference frame image area within the 
reference frame image' (see page 1 , lines 31 - 33; page 6, lines 1 2 - 22; page 7 lines 
14 - 25). Peleg specifically suggests that sub-regions of the images are selected or 
inclusion in the mosaic based on location or quality (predetermine rule); 

'a comparison reference extracting module for extracting one comparison 
reference frame image area from among the reference frame image area and the object 
frame image areas for synthesis' (see page 7, lines 14 - 25). Peleg specifically 
suggests that the automatic selection finds appropriate cut lines between neighboring 
images; 

'a target extracting module for extracting one target frame image area from 
among the object frame image areas for synthesis other than the comparison reference 
frame image area (see page 6, lines 7-10). Peleg specifically suggests that source 
selection includes cutting sub images (target frame area) out of larger images (object 
frame image); 

'a comparing module for comparing the comparison reference frame image area 
with the target frame image area to calculate a pre-selected parameter' (see page 3, 
lines 28 - 36; page 6, lines 16-19; page 6 lines 36 - page 7 line 4). Peleg specifically 
suggests that the alignment process provides information on the degree of overlap and 
that images may be discarded if their overlap is too large or images added if the overlap 
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is too little; the overlap corresponds to the pre-selected parameter and the alignment of 
the images teaches a comparison of those images; 

'an excluding module for excluding the target frame image area from the object 
frame image areas for synthesis if the parameter does not meet a predetermined 
criterion' (see page 6, lines 36 - page 7, line 4; page 28, lines 9-17). Peleg specifically 
suggests that when alignment fails, the image that is being aligned is changed 
(excluded) and that alignment failure can be detected by variety of criteria 
(predetermined criterion); and 

'a synthesized image generating module for synthesizing the reference frame 
image area and the object frame image areas for synthesis to create a synthesized 
image area' (see page 3, lines 28 - 36; page 6, lines 12-22; page 7, lines 33 - 35). 
Peleg specifically suggests the generation of a mosaic image (synthesized image area) 
from a set of images. 

12. Regarding claim 2, Peleg discloses 'a setting module for setting as the reference 
frame image area an area within the reference frame image, to serve as a reference for 
synthesis' (see page 3, lines 28 - 36). Peleg specifically suggests constructing a 
mosaic image by selecting source images, aligning them to select source image 
segments (frame image area within the frame image) to merge (synthesize) them into a 
mosaic; and 

'a frame number controlling module for repeating processes of the synthesis 
object setting module, the comparison reference extracting module, the object 
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extracting module, the comparing module, and the excluding module, until the total 
number of the reference frame image area and the object frame image areas for 
synthesis meeting the criterion reaches a predetermined number, (see page 9, lines 29 
- 34; page 5 line 35 - 35; page 6, lines 1 - 4). Peleg specifically suggests that his 
method is iterative this teaches that all steps setting, comparison, extraction and 
exclusion are repeated. Peleg also teaches that start and stop frames are selected and 
that all intermediate frames should be used, which teaches that once the number of 
intermediate frames is used (the predetermined number) processing stops. 

13. Regarding claim 3, Peleg discloses 'a specification receiving module for receiving 
specification of the reference frame image, wherein the setting module sets the 
specified frame image as the reference frame image' (see abstract; page 5 line 31 - 
34). Peleg specifically suggests that a set of images is selected to be combined from a 
set of source images and that the selection process finds a set of good quality images 
that cover the intended domain and content (reference frame image). 

14. Regarding claim 4, Peleg discloses 'the comparison reference extracting module 
sets the reference frame image area as the comparison reference frame image area' 
(see page 28, lines 19 - 26). Peleg specifically suggests that the most recently aligned 
image is designated the region of interest for correlation (comparison) between it and 
current image. 
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1 5. Regarding claim 5, Peleg discloses 'an eliminating module for eliminating, from 
among the object frame image areas for synthesis, an area of frame image for which a 
characteristic of the frame image area meets a predetermined condition' (see page 6, 
line 37 - page 7 line 4). Peleg specifically suggests that images may be discarded 
(eliminated) if their overlap is too large (predetermined condition). 

16. Regarding claim 6, Peleg discloses 'the parameter is an image shift amount' (see 
page 6, lines 16-19; page 6, line 37 - page 7 line 4). Peleg specifically suggests that 
the parameter may be shift, rotate, dilate, projective or general flow. 

17. Regarding claim 7, Peleg discloses 'a frame shift calculating module for 
calculating the image shift amount of a target frame image containing the target frame 
image area, with respect to a comparison reference frame image containing the 
comparison reference frame image area (see page 6, lines 12-19). Peleg specifically 
suggests that the source images are aligned with one another so that each is in 
registration with the corresponding portions of neighboring images and that alignment 
entails finding a geometrical transformation (shift) that brings them into a common 
coordinate system; By finding this geometrical transformation the shift amount of the 
target frame with respect to the comparison frame image is found as well as the shift 
amount of the target frame image area with respect to the comparison image area; and 

'an area shift calculating module for calculating the image shift amount of the 
target frame image area with respect to the comparison reference frame image area, 
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based on the image shift amount calculated by the frame shift calculating module'(see 
page 6, lines 12-19). Peleg specifically suggests that the source images are aligned 
with one another so that each is in registration with the corresponding portions of 
neighboring images and that alignment entails finding a geometrical transformation 
(shift) that brings them into a common coordinate system; By finding this geometrical 
transformation the shift amount of the target frame with respect to the comparison frame 
image is found as well as the shift amount of the target frame image area with respect 
to the comparison image area. 

18. Regarding claim 8, Peleg discloses 'wherein the parameter is an image 
difference derived from comparison of pixel characteristic values at identical locations in 
the target frame image area and the comparison reference frame image area' (see page 
1 1 , lines 21 - 25). Peleg specifically suggests the use of cross correlation and least 
square error as methods for aligning pairs of images and maximize a measure of match 
(difference derived from comparison) over the overlap regions (identical locations in the 
image areas). 

1 9. Regarding claim 1 1 , Peleg discloses ' An image generating device for generating 
a still image from a plurality of frame images contained in a video image' (see page 3, 
lines 28 - 36; Peleg specifically suggests an apparatus for constructing an image 
mosaic), 'wherein if an evaluation value based on a characteristic value of an image 
(see page 3, lines 28 - 36; page 6, lines 16-19; page 6 lines 36 - page 7 line 4; Peleg 
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specifically suggests that the alignment process provides information on the degree of 
overlap and that images may be discarded if their overlap is too large or images added 
if the overlap is too little; the overlap corresponds to the characteristic value), the 
evaluation value being calculated by comparing a reference frame image area which is 
an area serving as a reference for synthesis and contained in a reference frame image 
which is one of the plurality of frame images, with a comparison target frame image 
area which is one of areas contained in the plurality of frame images (see page 28, lines 
20 - 21 ; Peleg specifically suggests a doing correlation between an image and an 
region of interest), meets a predetermined criterion, the image generating device 
synthesizes the reference frame image area and the comparison target frame image 
area' (see page 3, lines 28 - 36; page 6, lines 16-19; page 6 lines 36 - page 7 line 4; 
page 7, lines 33 - 35) Peleg specifically suggests that the alignment process provides 
information on the degree of overlap and that images added. Peleg further suggests 
that after this step source images are combined into a single mosaic. 

20. Method claim 12 is drawn to the method of using the corresponding apparatus 
claimed in claim 1. Therefore method claim 12 corresponds to apparatus claims 1 and 
is rejected for the same reasons of anticipation as used above. 

21 . As to claim 13, the discussions are addressed with respect to claim 2. 



22. 



As to claim 14, the discussions are addressed with respect to claim 3. 



Application/Control Number: 10/541,479 Page 13 

Art Unit: 2624 

23. As to claim 15, the discussions are addressed with respect to claim 4. 

24. As to claim 16, the discussions are addressed with respect to claim 5. 

25. As to claim 17, the discussions are addressed with respect to claim 6. 

26. As to claim 18, the discussions are addressed with respect to claim 7. 

27. As to claim 19, the discussions are addressed with respect to claim 8. 

28. Method claim 22 is drawn to the method of using the corresponding apparatus 
claimed in claim 1 1 . Therefore method claim 22 corresponds to apparatus claims 1 1 
and is rejected for the same reasons of anticipation as used above. 

29. Regarding claim 23, Peleg discloses 'a computer program stored on a computer 
readable medium for generating a still image from a plurality of frame images contained 
in a video image" (see page 25, lines 27 - 36). Peleg specifically suggests that 
software running on a unmodified computer to perform a method to generate an image 
mosaic (a still image from a plurality of images) 

The remaining limitations of claim 23 are similar in scope to those of claim 1 as 
such they are addressed in the discussions of claim 1 . 
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30. Regarding claim 24, Peleg discloses 'A computer program stored on a computer 
readable medium for generating a still image from a plurality of frame images contained 
in a video image' (see page 25, lines 27 - 36). Peleg specifically suggests that software 
running on a unmodified computer to perform a method to generate an image mosaic (a 
still image from a plurality of images). 

The remaining limitations of claim 24 are similar in scope to those of claim 1 1 as 
such they are addressed in the discussions of claim 1 1 . 

Claim Rejections - 35 USC § 103 

31 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

32. Claims 9 - 1 0 and 20 - 21 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Peleg et al., WO 98/02844 published on Jan. 22, 1998 ("Peleg") as 
applied to claims 1 and 1 1 above, and further in view of Szeliski et al., U. S. Patent No. 
5,987,164 published Nov. 16, 1999 ("Szeliski"). 

33. Regarding claim 9, Peleg discloses 'an image generating device according to 
claim 1' (see page 3, line 32 -36 ; page 5, lines 10 - 15). Peleg specifically suggests 
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an apparatus for constructing an image mosaic (a still image from a plurality of images) 
and that the source of images can be a video camera (hence a video). It is noted that 
Peleg does not specifically disclose 'the parameter is a correlation of average values of 
pixel characteristic values in the target frame image area and in the comparison 
reference frame image area' 

However, within the same field of endeavor, Szeliski does disclose 'the 
parameter is a correlation of average values of pixel characteristic values in the target 
frame image area and in the comparison reference frame image area' (see column 26, 
lines 26 - 37). Szeliski specifically suggests that for each patch (target frame area and 
reference frame area) pair-wise motion estimates are combined in a normalized 
average (correlation of average values). 

It would have been obvious to one of ordinary skill in the art at the time of 
applicant's invention to incorporate the teachings of Szeliski in to those of Peleg, 
because both Szeliski and Peleg are within the same field of endeavor (see Peleg 
Abstract and Szeliski Abstract), both are directed to building image mosaics. Further, 
users of Peleg's invention would not only be able to generate mosaic images from 
video, but would also have the ability to post process such mosaics into a convenient 
viewing surface (see Szeliski column 3, lines 56 - 60) 

34. Regarding claim 10, Szeliski discloses 'the reference frame image area and the 
object frame image area for synthesis are areas derived by dividing each frame image 
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in an identical manner' (see abstract; column 20, lines 58 - 63). Szeliski specifically 
suggests dividing both images into a number of 16x16 patches; and 

'the target extracting module extracts a target frame image area at a same 
location corresponding to the comparison reference frame image area' (see column 4, 
lines 13-18). Szeliski specifically suggests that for each of the pixel locations in the 
one image, ray directions are determined to the relative 3-dimensional coordinate 
system of the corresponding pixel location (same location) in each of the other images. 

35. As to claim 20, the discussions are addressed with respect to claim 9. 



36. 



As to claim 21 , the discussions are addressed with respect to claim 10. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to PIERRE DIVERSE whose telephone number is 
(571 )270-391 1 . The examiner can normally be reached on Monday to Thursday 
8:00am - 5:00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Pierre Diverse/ 
Examiner, Art Unit 2624 

IP. D.I 

/Brian Q Lei 

Primary Examiner, Art Unit 2624 



